Ergodic BSDEs and Optimal Ergodic Control in Banach Spaces 



Marco Fuhrman 
Dipartimento di Matematica, Politecnico di Milano 
piazza Leonardo da Vinci 32, 20133 Milano, Italy 
e-mail: marco.fuhrman@polimi.it 

l> . 

O . Ying Hu 

IRMAR, Universite Rennes 1 
Campus de Beaulieu, 35042 RENNES Cedex, France 
e-mail: ying.hu@univ-rennesl.fr 



O 

(N 



00 . 

, Gianmario Tessitore 

Dipartimento di Matematica e Applicazioni, Universita di Milano-Bicocca 
P^'' Via Cozzi 53, 20135, Milano Italy 

Q_( . e-mail: gianmario.tessitore@unimib.it 

Abstract 

In this paper we introduce a new kind of Backward Stochastic Differential Equations, 
called ergodic BSDEs, which arise naturally in the study of optimal ergodic control. We 
J> ' study the existence, uniqueness and regularity of solution to ergodic BSDEs. Then we apply 

these results to the optimal ergodic control of a Banach valued stochastic state equation. 
We also establish the link between the ergodic BSDEs and the associated Hamilton- Jacobi- 
Bellman equation. Applications are given to ergodic control of stochastic partial differential 
• ' equations. 

o 

^ ; 1 Introduction 

In this paper we study the following type of (markovian) backward stochastic differential equa- 
tions with infinite horizon (that we shall call ergodic BSDEs or EBSDEs for short): 

r-T r-T 

Y f = Yf + J [i>(X%, ZD -\\da- Z*dW a , < t < T < oo. (1.1) 

In equation (jl.ip X x is the solution of a forward stochastic differential equation with values in 
a Banach space E starting at x and (Wt)t>o is a cylindrical Wiener process in a Hilbert space 

Our aim is to find a triple (Y, Z, A) , where Y, Z are adapted processes taking values in R and 
H* respectively and A is a real number, ifi : E x H* — > R is a given function. We stress the fact 
that A is part of the unknowns of equation (jl.ip and this is the reason why the above is a new 
class of BSDEs. 

It is by now well known that BSDEs provide an efficient alternative tool to study optimal 
control problems, see, e.g. [21], [9] or, in an infinite dimensional framework, [12], [17]. But up to 
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our best knowledge, there exists no work in which BSDE techniques are applied to optimal con- 
trol problems with ergodic cost functionals that is functionals depending only on the asymptotic 
behavior of the state (see e.g. the cost defined in formula (jl.4p below). 

The purpose of the present paper is to show that backward stochastic differential equations, 
in particular the class of EBSDEs mentioned above, are a very useful tool in the treatment of 
ergodic control problems as well, especially in an infinite dimensional framework. 

There is a fairly large amount of literature dealing by analytic techniques with optimal ergodic 
control problems for finite dimensional stochastic state equations. We just mention the basic 
papers by Bensoussan and Frehse [3] and by Arisawa and Lions pQ where the problem is treated 
through the study of the corresponding Hamilton-Jacobi-Bellman (HJB) equation (solutions are 
understood in a classical sense and in a viscosity sense, respectively). 

Concerning the infinite dimensional case it is known that both classical and viscosity notions 
of solutions are not so suitable concepts. Maslowski and Goldys in [15] employ a mild formulation 
of the Hamilton-Jacobi-Bellman equation in a Hilbertian framework (see [5] and references within 
for the corresponding mild formulations in the standard cases). In [15] the authors prove, 
by a fixed point argument that exploits the smoothing properties of the Ornstein-Uhlenbeck 
semigroup corresponding to the state equation, existence and uniqueness of the solution of the 
stationary HJB equation for discounted infinite horizon costs. Then they pass to the limit, as 
the discount goes to zero, to obtain a mild solution of the HJB equation for the ergodic problem 
(see also [8]). Such techniques need to assume, beside natural condition on the dissipativity of 
the state equation, also non-degeneracy of the noise and a limitation on the lipschitz constant 
(with respect to the gradient variable) of the hamiltonian function. This last condition carries 
a bound on the size of the control domain (see [14] for similar conditions in the infinite horizon 
case) . 

The introduction of EBSDEs allow us to treat Banach valued state equations with general 
monotone nonlinear term and possibly degenerate noise. Non-degeneracy is replaced by a struc- 
ture condition as it usually happens in BSDEs approach, see, for instance, [9], [12]. Moreover 
the use of L°° estimates specific to infinite horizon backward stochastic differential equations 
(see [1], [22], [16] ) allow us to eliminate conditions on the lipschitz constant of the hamiltonian. 
On the other side we will only consider bounded cost functionals. 

To start being more precise we consider a forward equation 

dX'f = {AX? + F{X x ))dt + GdW t , X = x 

where X has values in a Banach space E, F maps E to E and A generates a strongly continuous 
semigroup of contractions. Appropriate dissipativity assumptions on A + F ensure the exponen- 
tial decay of the difference between the trajectories starting from different points x, x' £ E. 
Then we introduce the class of strictly monotonic backward stochastic differential equations 

Y x,a = y x, a + [ T ^(x*, Z x > a ) - aY x ' a )da - [ T Z x ' a dW a , < t < T < oo. (1.2) 
Jt Jt 

for all a > (see [3], [22] or [16]) where ip : E x H* — > R is bounded in the first variable and 
Lipschitz in the second. By estimates based on a Girsanov argument introduced in [4] we obtain 
uniform estimates on aY x ' a and Y x ' a — Y x ,a that allow us to prove that, roughly speaking, 
(Y x ' a - Y 0,a , Z x ' a , aY®' a ) converge to a solution (Y x , Z x , A) of the EBSDE ([TT]) . for all x <= E. 
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We also show that A is unique under very general conditions. On the contrary, in general we can 
not expect uniqueness of the solution to (jl.ip , at least in the non markovian case. On the other 
side in the markovian case we show that we can find a solution of (jl.ip with Y t x = v(Xf) and 
Zf = ((Xf) where v is Lipschitz and t>(0) = 0. Moreover (v,Q are unique at least in a special 
case where tp is the Hamiltonian of a control problem and the processes X x are recurrent (see 
Section [8] where we adapt an argument from [15] ) . 

If we further assume differentiability of F and ip (in the Gateaux sense) then v is differen- 
tiable, moreover £ = VvG and finally (v, A) give a mild solution of the HJB equation 

Cv(x) + ip (x, Vv(x)G) = A, xeE, (1.3) 

where linear operator C is formally defined by 

Cf (x) = X -Trace {GG*V 2 f (x)) + (Ax, V/ (x)) e ,e> + {F (*) , V/ (x)) EiE *. 

Moreover if the Kolmogorov semigroup satisfies the smoothing property in Definition 15.11 and F 
is genuinely dissipative (see Definition I5.2|) then v is bounded. 

The above results are then applied to a control problem with cost 

i r T 

J(x,u) = limsup — E / L(X x ,u s )ds, (1.4) 
T->oo T J 

where u is an adapted process (an admissible control) with values in a separable metric space 
U, and the state equation is a Banach valued evolution equation of the form 

dXf = (AXf + F(Xf)) dt + G(dW t + R(u t ) dt), 

where R : U — > S is bounded. It is clear that the above functional depends only on the 
asymptotic behavior of the trajectories of X x . After appropriate formulation we prove that, 
setting ip(x,z) = mi u£ u[L(x,u) + zR(u)\ in (II. ip . then A is optimal, that is 

A = inf J(x, u) 

u 

where the infimum is over all admissible controls. Moreover Z allows to construct on optimal 
feedback in the sense that 

A = J(x, u) if and only if L(Xf ,u t ) + Z t R{u t ) = ip(X? , Z t ). 

Finally, see Section O we show that our assumptions allow us to treat ergodic optimal 
control problems for a stochastic heat equation with polynomial nonlinearity and space-time 
white noise. We notice that the Banach space setting is essential in order to treat nonlinear 
terms with superlinear growth in the state equation. 

The paper is organized as follows. After a section on notation, we introduce the forward SDE; 
in section 4 we study the ergodic BSDEs; in section 5 we show in addition the differentiability 
of the solution assuming that the coefficient is Gateaux differentiable. In section 6 we study the 
ergodic Hamilton-Jacobi-Bellman equation and we apply our result to optimal ergodic control 
in section 7. Section 8 is devoted to show the uniqueness of Markovian solution and the last 
section contains application to the ergodic control of a nonlinear stochastic heat equation. 
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2 Notation 



Let E, F be Banach spaces, H a Hilbert space, all assumed to be defined over the real field and 
to be separable. The norms and the scalar product will be denoted | • |, ( • , • ), with subscripts if 
needed. Duality between the dual space E* and E is denoted ( • , • )e*,e- L(E, F) is the space of 
linear bounded operators E — > F, with the operator norm. The domain of a linear (unbounded) 
operator A is denoted D(A). 

Given a bounded function 4> : E — > R we denote \\4>\\o = sup x£E |</>(x)|. If, in addition, 4> is 
also Lipschitz continuous then ||0||]jp = ||<^||o + su Px,x'ge,x^x' ~~ </>(x')||x — x'| _1 . 

We say that a function F : E — > F belongs to the class Q 1 (E,F) if it is continuous, has a 
Gateaux differential X7F(x) G L(E, F) at any point x G E, and for every k G E the mapping 
x — ¥ VF(x)k is continuous from E to F (i.e. x — > X7F(x) is continuous from E to L(E, F) if the 
latter space is endowed the strong operator topology). In connection with stochastic equations, 
the space Q 1 has been introduced in [12], to which we refer the reader for further properties. 

Given a probability space (O, T, P) with a filtration (J-t)t>o we consider the following classes 
of stochastic processes with values in a real separable Banach space K. 

1. Lj,(Q, C([0, T], K)), p G [1, oo), T > 0, is the space of predictable processes Y with 
continuous paths on [0, T] such that 

l y l^(n A [0,T], S ))= E ^P T] l y t^< 00 - 

2. L P v (VL,L 2 {[0,T]-K)), p G [1,oo), T > 0, is the space of predictable processes 7 on [0, T] 
such that 

r T \ p/2 

< oo . 



3. L2, loc (fi;L 2 (0,oo;K)) is the space of predictable processes Y on [0, oo) that belong to the 
space L^(Q, L 2 ([0, T\;K)) for every T > 0. 

3 The forward equation 

In a complete probability space (Q,^-", P) , we consider the following stochastic differential equa- 
tion with values in a Banach space E: 

dX t = AX t dt + F{X t )dt + GdW t , t>0, 
X = x G E. 

We assume that E is continuously and densely embedded in a Hilbert space H , and that both 
spaces are real separable. 

We will work under the following general assumptions: 

Hypothesis 3.1 1. The operator A is the generator of a strongly continuous semigroup of 
contractions in E. We assume that the semigroup {e tA , t > 0} of bounded linear operators 
on E generated by A admits an extension to a strongly continuous semigroup of bounded 
linear operators on H that we denote by {S(t), t > 0}. 

2. W is a cylindrical Wiener process in another real separable Hilbert space 3. Moreover 
by Tt we denote the a -algebra generated by {W s , s G [0,t]} and by the sets of T with 
^-measure zero. 
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3. F : E — > E is continuous and has polynomial growth (that is there exist c > 0, k > such 
that \F(x)\ < c(l + \x\ k ), x G E). Moreover there exists r\ > such that A + F + rjl is 
dissipative. 

4- G is a bounded linear operator from E to H . The bounded linear, positive and symmetric 
operators on H defined by the formula 

Q t h= [ S(s)GG*S*{s)hds, t>0,heH, 
Jo 

are assumed to be of trace class in H. Consequently we can define the stochastic convolution 

[ S(t - s)GdW s , t > 0, 
J o 



W A 



as a family of H-valued stochastic integrals. We assume that the process {W t A , t > 0} 
admits an E -continuous version. 

We recall that, for every x G E, with x ^ 0, the sub differential of the norm at x, d(\x\), is 
the set of functional x* G E* such that (x*,x) E „ E = \x\ and |x*|_e* =1. If x = then 9(|a;|) 
is the set of functionals x* G E* such that |a;*|£:* < 1. The dissipativity assumption on A + F 
can be explicitly stated as follows: for x,x' G D(A) C E there exists x* G d(\x — x'\) such that 

(x*,A(x -x')+ F (x) - F [x') ) E# E < -7] \x - x'\ . 

We can state the following theorem, see e.g. 16], theorem 7.13 and [7], theorem 5.5.13. 

Theorem 3.2 Assume that Hupothesis \3 . 1\ holds true. Then for every x G E equation $3. 1\) ad- 
mits a unique mild solution, that is an adapted E-valued process with continuous paths satisfying 
P-a.s. t t 

X t = e tA x+ [ e {t - s)A F{X s )ds+ [ e {t - s)A GdW s , t > 0. 
Jo Jo 

We denote the solution by X x , x G E. 

Now we want to investigate the dependence of the solution on the initial datum. 
Proposition 3.3 Under Hypothesis I3.il it holds: 

{X? 1 - X? 2 1 < e _r?t \ Xl - x 2 \ , t > 0, xx,x 2 e E. 

Proof. Let X x (t) = X* 1 and X 2 (t) = X? 2 , xx,x 2 G E. For i = 1,2 we set X? (t) = J n X t (t), 
where J n = n (nl — A)~ . Since X" (t) G D (A) for every t > 0, and 

X? (t) = e tA J nXl + f e^ A J n F (X t (*)) ds + f e^ A J n GdW s , 
Jo Jo 

we get 

| (X? (t) - X 2 " (t)) = ^ (X? (t) - X 2 " (i)) + J n [F (X, (t)) - F (X 2 (t))] . 

So, by proposition II. 8. 5 in }24j also |X™ (t) — X% (i) | admits the left and right derivatives 
with respect to t and there exists x* n (t) G d (|-X~" (t) — X 2 (t)\) such that the left derivative 
of \Xf (t) - X% (t)\ satisfies the following 

d - |Xf (t) - X n 2 (t)\ = (x* n (t) , j t (Xf (t) - X n 2 (i))^ ^ . 
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So we have 

^ |X? (t) - X% (i)| = « (t) , A (Xf (i) - X 2 ™ (t)) + F (Xf (t)) - F (X 2 " (t))) E *,E 

+ (x* n (t) , J n F {X x (t)) - F (X? (t))) E ;E 
- (x* n (i) , J n F (X 2 (t)) - F (X 2 " 
< -»/ |AT (t) - -X? (*)| + 1*5? (t) - «55 (i)|, 

where for i = 1, 2 we have set o~™ (t) = J n F (X; (i)) - F (Xf (f)). 
Multiplying the above by e vt we get 

d — (e* |Xf (t) - X- (t)\) < |d» (t) - % (t)| . 

We note that <5" (t) tends to uniformly in i £ [0, T] for arbitrary T > 0. Indeed, 

<5? (t) = nF (n, A) [F (X, (i)) - F (Xf (t))} + (nF (n, A) - I) F (X (i)) , 

and the convergence to follows by a classical argument, see e.g. the proof of theorem 7.10 in [6j, 
since Xf (t) tends to Xj (t) uniformly in t E [0, T] and the maps t h- > Xj (£) and i i— > F (Xj (£)) 
are continuous with respect to i. 

Thus letting n->oowe can conclude 

|X!(i)-X 2 (i)| <e-^|si-x 2 |- 

and the claim is proved. ■ 

We will also need the following assumptions. 
Hypothesis 3.4 We have sup 4>0 EIW7/I 2 < oo. 

r+oo 

Hypothesis 3.5 e tA G (3) C F /or all t > and / |e* A G| L(Hi£;) dt < oo. 

J o 

We recall that for arbitrary gaussian random variabile Y with values in the Banach space 

F, the inequality 

E(p(\Y\ - E |F|) < E^(2A/E|y| 2 7 ) 

holds for any convex nonnegative continuous function on F and for 7 a real standard gaussian 
random variable, see e.g. [10] , Example 3.1.2. Upon taking ^(x) = \x\ p , it follows that for every 
p > 2 there exists c p > such that E \ Y \ p < c p (E |Y | 2 ) p / 2 . By the gaussian character of W A and 
the polynomial growth condition on F stated in Hypothesis 13. 11 point 3, we see that Hypothesis 
13.41 entails that for every p > 2 

supE [\W t A \ p + \F(W t A )\ p ] < 00. (3.2) 
t>o 

Proposition 3.6 Under Hypothesis \3.1\ it holds, for arbitrary T > and arbitrary p > 1 

E sup |Xf | p < C PjT (l + \x\ p ), x G F. (3.3) 
te[o,T] 

//, m addition, Hypothesis \3.4\ holds then, for a suitable constant C 

supE|Xf| < C(l + \x\), x£E. (3.4) 
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Moreover if, in addition, Hypothesis \3.5\ holds, 7 is a bounded, adapted, E-valued process and 
X X '' Y is the mild solution of equation 



dX; a = AX;> 7 dt + F{X* n )dt + GalW t + Gj t dt, t > 0, 



X^ = xeE. 



(3.5) 



then it is still true that 



supE|^' 7 | < C 7 (l + \x\), x € E, (3.6) 
t>o 



for a suitable constant C 7 depending only on a uniform bound for 7. 
Proof. We let Z t = X? - W t A , Z t n = J n Z t , then 

j t Z? = AZ? + J n F(Xf ) = AZ? + [F(Z? + J n W t A ) - F{J n W t A )} + F(W t A ) + ^ 

where 

^ = J n F(X?) - F{J n X?) + F(J n W A ) - F(W A ). 

Proceeding as in the proof of Proposition 13.31 observing that, for all t > 0, / |<5™ \ds — > 

Jo 

n — > 00, we get: 

\Z t \ < e-*\x\ + f e^ {t - s) \F{W A )\ds, P - a.s. 
Jo 



as 



and ([33]) follows from (|3T2jh 

In the case in which X x is replaced by X x ~ f the proof is exactly the same just replacing W A 
by W t Al = W A + f* e^^Gjsds. 

Finally to prove f|3.3|) we notice that (see the discussion in [T7j ) the process W A is a Gaussian 
random variable with values in C([0,T], E). Therefore by the polynomial growth of F we get 

E sup [\W A \P + \F(W A )\P]<C p , T (l + \xn, 
te[o,T] 

and the claim follows as above. ■ 

Finally the following result is proved exactly as Theorem 6.3.3. in [7]. 

Theorem 3.7 Assume that Hypotheses \3.1\ and \3.4\ hold then equation \3. 1\) has a unique in- 
variant measure in E that we will denote by \x. Moreover \i is strongly mixing (that is, for all 
x € E, the law of Xf converges weakly to fi as t — > 00). Finally there exists a constant C > 
such that for any bounded Lipschitz function <ft : E — » R, 



E0pffO- / 4>dfi <C(l + \x\)e-^ 2 U\\ l[p . 

J E 



4 Ergodic BSDEs (EBSDEs) 

This section is devoted to the following type of BSDEs with infinite horizon 

Y t x = YS + f mx x , ZD -\\da- [ T 2% dW a , 0<t<T<oo, (4.1) 
Jt Jt 

where A is a real number and is part of the unknowns of the problem; the equation is required to 
hold for every t and T as indicated. On the function ip : E x H* — > R and assume the following: 
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Hypothesis 4.1 There exists K X ,K Z > such that 

\ip(x, z) — tp(x' , z)\ < K x \x — x \ + K z \z — z |, x, x G E, z, z G S*. 
Moreover - ,0) is bounded. We denote sup xGE |^(x,0)| fry M. 

We start by considering an infinite horizon equation with strictly monotonic drift, namely, for 
a > 0, the equation 

y x,a = y x, a + [ T ^(x*, Z^ a ) - aY x ' a )da - f Z x ' a dW a , < t < T < oo. (4.2) 

The existence and uniqueness of solution to (|4.2p under Hypothesis 14. II was first studied by 
Briand and Hu in [3] and then generalized by Royer in [22] . They have established the following 
result when W is a finite dimensional Wiener process but the extension to the case in which W 
is a Hilbert- valued Wiener process is immediate (see also [16J). 



Lemma 4.2 Let us suppose that Hypotheses I3.il and hold. Then there exists a unique 
solution (Y x,a ,Z x,a ) to BSDE such that Y x,a is a bounded continuous process, and Z x,a 

belongs to Lp loc (0; L 2 (0, oo; H*)). 

Moreover '\Y x ' a \ < M/a, F-a.s. for all t > 0. 

We define 

v a (x) = Y a ^. 

We notice that by the above |v a (x)| < M/a for all x G E. Moreover by the uniqueness of the 
solution of equation (|4.2p it follows that Y"' x = v a {Xf) 

To establish Lipschitz continuity of v a (uniformly in a) we use a Girsanov argument due to 
P. Briand and Y. Hu, see [3]. Here and in the following we use an infinite-dimensional version 
of the Girsanov formula that can be found for instance in [6]. 

Lemma 4.3 Under Hupotheses \3.1\ and \4-l\ the following holds for any a > 0: 



\v a {x) -v a {x')\ < — \x-x'\, x,x'eE. 
V 

Proof. We briefly report the argument for the reader's convenience. 
We set Y = Y a ' x - Y a > x ' , Z = Z a ' x - Z c 



7Q,:r 



wxf,z?< x )-i>(xf,zr) ( 7 *,* _ w 



0, elsewhere, 

f t = ^xf,zr)-^xf,zr). 

By Hypothesis 14.11 (5 is a bounded H-valued, adapted process thus there exists a probability P 
under which W% = J * j3 s ds + Wt is a cylindrical H-valued Wiener process for t G [0, T], Then 
(y, Z) verify, for all < t < T < oo, 

Y t = Y T -a f %da + / f a da - [ Z a dW a . (4.3) 
Jt Jt Jt 
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Computing d(e at Y t ), integrating over [0, T], estimating the absolute value and finally taking 
the conditional expectation E^' with respect to P and Tt we get: 

\%\ < e- a ( T -*)t^|y T | +E Fi J T e- a ^\f s \ds 

Now we recall that Y is bounded and that \ft | < K x \Xf — X x \ < K x e~ rit \x — x'\ by Proposition 
13.31 Thus if T — > oo we get |Yj| < K x {r\ + a)~ 1 e at \x — x'\ and the claim follows setting t = 0. ■ 

By the above Lemma if we set 

jf( x ) =v a (x) -v a (0), 

then |TJ a (x)| < K^ -1 ^] for all x E E and all a > 0. Moreover by Lemma H~2l q|^ a (0)| < M. 
Thus by a diagonal procedure we can construct a sequence a n \ such that for all x in a 
countable dense subset D C E 

v an (x)^v{x), a n v an (0)^\, (4.4) 

for a suitable function v : D — > R and for a suitable real number A. 

Moreover, by Lemma 14.31 \v a {x) — v a (x')\ < K x r]~ l \x — x'\ for all x,x' £ E and all a > 0. 
So u can be extended to a Lipschitz function defined on the whole E (with Lipschitz constant 
K x r] ) and 

v a "(x) -> u(x), ie£ (4.5) 

Theorem 4.4 Assume Hvpotheses \3.1\ and \4.1\ hold. Moreover let X be the real number in ft4-4\ ) 
and define Yf = v{Xf) (where v is the Lipschitz function with v(0) = defined in ( f^.5| )j. Then 
there exists a process Z x E L^ loc (O;L 2 (0,oo;E*)) such that P-a.s. the EBSDE g^j is satisfied 
by (Y X ,Z X ,X) for allO <t <T. _ _ 

Moreover there exists a measurable function £ : E — > H* smc/i i/iaf = £(Xf ). 



Proof. Let yjf' a = y*' Q - u a (0) = v a (Xf ). Clearly we have, P-a.s., 

fT _ rT 

y x,a = y x, a + y y,(x2, Z x > a ) - aY x f - av a {0))da - J Z x ' a dW a , < t < T < oo. (4.6) 

Since |v a (x)| < K x \x\/rj, inequality (|3.3p ensures that Esup tg [ 0T ] [ su P«>o l^'^l 2 ] < +°° f° r 
any T > 0. Thus, if we define y x = v(X x ), then by dominated convergence theorem 

r T — — 

I \Y x t ,an -Y x t \ 2 dt^0 and E\Y^ a " - Y X T \ 2 -» 
•/ o 

as n — > oo (where a n \ is a sequence for which (|4.4p and (|4.5p hold). 
We claim now that there exists Z E L|> loc (i~2; L 2 (0, oo; 3*)) such that 

e f T \z*> an -z x \Ldt ^0 

Let y = P' a " — y a '- Qm , Z = Z x ' an — Z x ' am . Applying Ito's rule to Y 2 we get by standard 
computations 

y 2 + E f \Z t \Ldt = El^ + 2E f faYtdt - 2E f [a n Y t x ' an - a m Y t x ' am ] % dt 
Jo Jo Jo 
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where Vt = ip(X? , 2%'°") - tp(Xf , Z x ' am ). We notice that |^| < K z \Z t \ and a n \Y t x ' an \ < M. 
Thus 

e/ |Z t ||.dt<c E(lf) 2 + E I (Yf) 2 dt + E ( \Yf\dt 
Jo [ Jo Jo 

It follows that the sequence {Z x,am } is Cauchy in L 2 (Q; L 2 (0, T; H*)) for all T > and our claim 
is proved. 

Now we can pass to the limit as n — > oo in equation (|4.6p to obtain 



= y T + / MKXa) - A)da - / Z^dW CT , < t < T < oo. (4.7) 
Jt Jt 

We notice that the above equation also ensures continuity of the trajectories of Y It remains 
now to prove that we can find a measurable function £ : E — > H* such that Z t = Q(Xf), P-a.s. 
for almost every t > 0. 

By a general argument, see for instance [11 j . we know that for all a > there exists C, a : 
E —> E* such that Zf* = ( a (Xf), P-a.s. for almost every t > 0. 

To construct ^ we need some more regularity of the processes Z x,a with respect to x. 
If we compute d(Y^' a — Y^ ,a ) 2 we get by the Lipschitz character of ip: 



E 



I \Z?' a - zf' a \Ldt < E(v a (X£) - v a {xf)) 2 

T 

+E^ (K X \X X S - Xf\ + K z \Zf a - Zf> a \j \v a {X*) - v a (xf) 



By the Lipschitz continuity of v a (uniform in a) that of and Proposition 13.31 we immediately 
get: 

E f T \Z?' a - zf> a \Ldt < c\x - x'\ 2 . (4.8) 
Jo 

for a suitable constant c (that may depend on T). 

Now we fix an arbitrary T > and, by a diagonal procedure (using separability of E) we 
construct a subsequence (a' n ) C (a n ) such that a' n \ and 



E 



r \z?> a - - zf' a ™\i,dt<2- n 

Jo 



for all m > n and for all x € E. Consequently Z x ' an — > Z t , P-a.s. for a.e. t E [0,T]. Then we 
set: 

lim n £ a ™(x), if the limit exists in H*, 
0, elsewhere. 



C(x) 



Since = C n (X?) -» P-a.s. for a.e. t e [0, T] we immediately get that, for all x € E, the 

process Xf belongs P-a.s. for a.e. t € [0, T] to the set where lim n Q an (x) exists and consequently 
X ,i.Y/'). ■ 

Remark 4.5 We notice that the solution we have constructed above has the following "linear 
growth" property with respect to X: there exists c > such that, P-a.s., 

\Y X \ < c\Xf\ for all t > 0. (4.9) 

If we require similar conditions then we immediately obtain uniqueness of A. 
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Theorem 4.6 Assume that, in addition to Hvpotheses \3.1l \3.4\ and \4-l\ Hvpothesis \3. 51 holds as 
well. Moreover suppose that, for some x € E, the triple (V , Z' , A') verifies F-a.s. equation 
for all < t < T, where Y' is a progressively measurable continuous process, Z' is a process in 
L^ loc (ft;L 2 (0,oo;H*)) and X' € M. Finally assume that there exists c x > (that may depend 
on x) such that F-a.s. 

\Y{\ < c x (\Xf\ + 1), for all t > 0. 

Then A' = A. 

Proof. Let A = A' — A, Y = Y' — Y x , Z = Z' — Z x . By easy computations: 



T i-T 

Yt — Yq 



+ T- 1 I Zatdt-T- 1 I Z { d\Y, 



where 



It 



0, elsewhere , 



is a bounded H-valued progressively measurable process. By the Girsanov Theorem there exists 
a probability measure P 7 under which = — f^jsds + Wt, t G [0, T], is a cylindrical Wiener 
process in H. Thus computing expectation with respect to P 7 we get 



A = T _1 E P ^ 



Y T -Y 



Consequently, taking into account (|4.9h . 

|A| < cT-^QX^l + 1) + cT- 1 ^ + 1) (4.10) 
With respect to W" 1 , X x is the mild solution of 



dX^ = AX^dt + F{X^)dt + GdW? + Gj t dt, t>0 
Xq ' 7 = x G E. 

and by (13.6|) we get sup T>0 E p t \X%\ < oo. So if we let T — > oo in (I4.10|) we conclude that A = 0. 



Remark 4.7 The solution to EBSDE (|4.ip is, in general, not unique. It is evident that the 
equation is invariant with respect to addition of a constant to Y but we can also construct an 
arbitrary number of solutions that do not differ only by a constant (even if we require them to 
be bounded). On the contrary the solutions we construct are not Markovian. 
Indeed, consider the equation: 

- dY t = [^(Z t ) - \}dt - Z t dW t . (4.11) 

where W is a standard brownian motion and ip : R — > R is differentiable bounded and has 
bounded derivative. 

One solution is Y = 0;Z = 0; A = ^(0) (without loss of generality we can suppose that 

V>(o) = o). 

Let now <fi : R — ► R be an arbitrary differentiable function bounded and with bounded 
derivative. The following BSDE on [t, T] admits a solution: 



-dY s x ' 1 = ^{Z x s > % )ds-Z x s > l dW s , 
r 



Y*'* = 4>{x + W T -W t ). 
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If we define u(t, x) = Yf'* then both it and Vu are bounded. Moreover if Y% = Y®'° = 
u(t, W t ), Z t = = Vu{t, W t ) then 

( -dY t = iP(Z t )dt - Z t dWt, te[0,T], 
\ Y T = cj>{W T ). 

Then it is enough to extend with Yt = Yt, Zt = for t > T to construct a bounded solution to 

(BHD. 

Remark 4.8 The existence result in Theorem 14.41 can be easily extended to the case of ifi only 
satisfying the conditions 

\tP(x,z)-?P(x',z)\<K x \x-x'\, |V(x,0)|<M, \iP(x,z)\ <K z (1 + \z\). 

Indeed we can construct a sequence {tjj n : n G N} of functions Lipschitz in x and z such that for 
all x,x' G H, z G E*, n G N 

\^ n (x,z)-^ n (x',z)\<K'\x-x'\; \ip n (x, 0)1 < M'; lim \ijj n (x, z) - ip(x, z)\ = 0. 

n— >oo 

This can be done by projecting x to the subspaces generated by a basis in H* and then regular- 
izing by the standard mollification techniques, see [13]. We know that if (Y x ' n , Z x ' n , X n ) is the 
solution of the EBSDE gH) with tp replaced by i\) n then Y t x ' n = v n {Xf) with 

\v n {x) -v n {x')\ < ^ \x-x'\; v n (0)=0; \X n \ < M' 
V 

Thus we can assume (considering, if needed, a subsequence) that v n {x) — > v{x) and A n —* A. 
The rest of the proof is identical to the one of Theorem 14.41 

5 Differentiability 

We are now interested in the differentiability of the solution to the EBSDE (|4.1|) with respect 
to x. 

Theorem 5.1 Assume that Hypotheses I3.il and \4-l\ hold. Moreover assume that F is of class 
Q 1 (E,E) with\7F bounded on bounded sets ofE. Finally assume thattp is of class Q 1 (Ex3* , E). 
Then the function v defined in |^.5[ ) is of class ^ 1 (^,M). 

Proof. In [17] it is proved that for arbitrary T > the map x — » X x is of class Q l from E to 
Fp(£l, C ([0,T], E)) . Moreover Proposition 13.31 ensures that for all /i G E, 

|VXf h\ < e' vt \h\, P-a.s., for all t G [0,T]. (5.1) 

Under the previous conditions one can proceed exactly as in Theorem 3.1 of [16] to prove that 
for all a > the map v a is of class Q l . 

Then we consider again equation (14. 2h : 

Y x,a = yx,a + [ T ^(x x , Z x > a ) - aY x > a )da - [ T Z x > a dW a , < t < T < oo, 
Jt Jt 
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we recall that Y^' a = v a (X^), and apply again [T7] (see Proposition 4.2 there) and |12j (see 
Proposition 5.2 there) to obtain that for all a > the map x — > Y x > a is of class Q 1 from £J 
to L^(O,C([0,T],R)) and the map x -> Z x > a is of class G 1 from £ to L^,(VL, L 2 ([0, T],E*)). 
Moreover for all h £ E 1 it holds (for all £ > since T was arbitrary) 

-dVY"' x /i = [V^(Xf , Z?> x )VX?h + V z i/j(X?,Z?' x )VZ?' x h - aVY t a ' x h]dt - VZ a ' x hdW t . 
We also know that \Y t a ' x \ < M/a. Now we set 

jja,x = e vt VYt a ' x h, V a ' x = e^VZ^h. 
Then (u a ' x ,V a ' x ) satisfies the following BSDE: 

-dU^ x = [e^V x flf(Xf, Z? x )VXf - (a + r,)U^ x + V Z ^{X X , Z?' x )V t a > x ]dt - V t a ' x dW t . 
By (15. ip and the usual Girsanov argument (recall the V X V an d V z ip are bounded), 
\U?' X \ < — ^— , Vt > 0, P-a.s. i.e. \VY?' a \ < e - " 4 - 



a + r/ a + r] 

Moreover, consider the limit equation, with unknown (U X ,V X ), 

- dJJ'f = [e^Vx^OX?, Z X )VX X - 7]U t x + V z ip(X x , Z x )V x ]dt - V x dW t , (5.2) 

which, since \e nt 'V x 'il>\ 7 x Xf | is bounded, has a unique solution such that U x is bounded and V x 
belongs to Lj, loc (Q; L 2 (0, oo; H*)) (see [4] and [22J). 
We know that for a suitable sequence a n \ 0, 

and we claim now that 

W*"(x) = S/Y x ' an = U x ' an -> C/£. 
To prove this we introduce the finite horizon equations: for t € [0, N], 

_ dU x,a,N = [e vt Vx7 p( X ^ Z x ' a )VXf - (a + v )Uf ' a ' N + V z tl>(Xf, Z x ' a )V x ' a ' N ]dt 



T rX,a,N TTT7 

-V t dW t , 



jjX,a,N 
N 



J -dU?' N = [e^V x ^{Xf , Zf )VX? - (a + r/)C/f + V^(Xf, )Vf ^di - tf'^, 
1 U% N = 0. 

r-N 

Since E / \Z x > a " - Z x \ 2 ds -> it is easy to verify that, for all fixed iV > 0, U^' an ' N -» l^. 
•/ o 

On the other side a standard application of Girsanov Lemma gives see 116) . 



jjX,a n ,N _ v x,a n | < _^_ e -„7V ) ^.JV _ ^ | < ^-„iV 



e 



a n + T) u r/ 

for a suitable constant c. 

Thus a standard argument implies Uq' " — > U§. An identical argument also ensures conti- 
nuity of Uq with respect to x (also taking into account I4.8j) . The proof is therefore completed. 



As usual in the theory of markovian BSDEs, the differentiability property allows to identify 
the process Z x as a function of the process X x . To deal with our Banach space setting we need 
to make the following extra assumption: 
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Hypothesis 5.2 There exists a Banach space So, densely and continuously embedded in 3, such 
that G (Ho) C H and G : Ho — > E is continuous. 

We note that this condition is satisfied in most applications. In particular it is trivially true 
in the special case E = H just by taking So = H, since G is assumed to be a linear bounded 
operator from H to H. The following is proved in [17} Theorem 3.17]: 

Theorem 5.3 Assume that Hupotheses \ 3.1l \4-l\ and \5.S\ hold. Moreover assume that F is of 
class G 1 (E, E) with VF bounded on bounded subsets of E and ip is of class G 1 (E x 3*, E). Then 
Zf = Vv{Xf )G, P-a.s. for a.e. t > 0. 

Remark 5.4 We notice that Vw(x)G£ is only defined for £ G Ho in general, and the conclusion of 
Theorem [5]3] should be stated more precisely as follows: for £ G Ho the equality Zf £ = Vv(Xf)G£, 
holds P-a.s. for almost every t > 0. However, since Z x is a process with values in 3*, and more 
specifically a process in L|>(fi, L 2 ([0, T], 3*)), it follows that P-a.s. and for almost every t the 
operator £ — > Vv{Xf)G^ can be extended to a bounded linear operator defined on the whole 
3. Equivalently, for almost every t and for almost all x 6 E (with respect to the law of Xt) 
the linear operator £ — > Vv(x)G£ can be extended to a bounded linear operator defined on the 
whole 3 (see also Remark 3.18 in |17j). 

Remark 5.5 The above representation together with the fact that v is Lipschitz with Lipschitz 
constant K x rj~ l immediately implies that, ifFis of class G l (E, E) and ip is of class C/ 1 (£'x3*, E), 
then \Zf |n* < K x V~ 1 \G\l('E. ,e) f° r an x & E, P-a.s. for almost every t > 0. Consequently we can 
construct C, in Theorem 14.41 in such a way that it is bounded in the 3q norm by K x t]~ 1 \G\l(z ,e)- 
Once this is proved we can extend the result to the case in which ip is no longer differentiable 
but only Lipschitz, namely we can prove than even in this case the process Z x is bounded. 
Indeed if we consider a sequence {ip n : n € N} of functions of class G 1 (E x 3*,E) such that for 
all x, x 1 G H, z, z' G 3*, n G N, 

\ij) n {x,z) - if) n (x',z')\ < K x \x - x'\ + K z \z - z'\; lim \ip n (x, z) - ip(x, z)\ = 0. 

n— >oo 

We know that if (Y x ' n , Z x ' n , X n ) is the solution of the EBSDE (|4.1|) with ip replaced by ip n then 
l^' n |=* — ^x r ]^ 1 \G\L(E ,E)- Then as we did above we can show (showing that the corresponding 
equations with monotonic generator converge uniformly in a) that E / Q \Z*' n - Zf %*dt -> 
and the claim follows. 

We also notice that by the same argument we also have |C Q (^)|s2 < KxV~ 1 \G\l(~. q ,e), ^ a > 0- 

Now we introduce the Kolmogorov semigroup corresponding to X: for measurable and bounded 
(j) : E — > R we define 

P t [<t>]{x) =K(j)(X x ) t>0,xeE. (5.3) 

Definition 5.1 The semigroup (Pt)t>o is called strongly Feller if for all t > there exists kt 
such that for all measurable and bounded <f> : E — ► R ; 

\P t [4>](x) - P t [(t>]{x')\ < fetll^llolx-a/l, x,x' G E, 

where \\<f>\\ = sup xeE \<p(x)\. 

This terminology is somewhat different from the classical one (namely, that Pt maps mea- 
surable bounded functions into continuous ones, for all t > 0), but it will be convenient for 
us. 
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Definition 5.2 We say that F is genuinely dissipative if there exist e > and c > such that, 
for all x,x' 6 E, there exists z* 6 d\x — x'\ such that < z*,F(x) — F(x') >e*,e< c\x — 

Lemma 5.6 Assume that Hypotheses lff.il and \3.4\ hold. If the Kolmogorov semigroup (Ft) is 
strongly Feller then for all bounded measurable (j) : E — > M., 



PMUx 



4>(x)fi(dx) 



If in addition F is genuinely dissipative then 



< ce 



1 + \x 



PtMx)- / <t>{x)ii{dx) 



< ce 



Proof. We fix e > 0. For t > 2 we have, by Theorem 13.71 



PM(x 



4>(x)fi(dx) 



Pt-l[P\ 



P 1 [(j)]{x)ii{dx) 



<C(l + \x\)e-^\\PM\\\i v 
<C(l + |s|)e-^ 4 fci||0||o, 



and the first claim follows since Pt[(^](x) — f E 4>(x) /j(dx)\ < 2||0||o. 

If now F is genuinely dissipative then in [7j, Theorem 6.4.1 it is shown that 



(ftdfj. 



< Ce-^l 2 



lip 



and the second claim follows by the same argument. ■ 

We are now able to state and prove two corollaries of Theorems 15.11 and [57 



Corollary 5.7 Assume that Hypotheses \3.1l \3.4\ \4-l\ and \5.Si hold. Moreover assume that F is 
of class Q l with S/F bounded on bounded subsets of E, and that ip is bounded on each set E x B, 
where B is any ball o/Hq. Finally assume that the Kolmogorov semigroup (Pt) is strongly Feller. 
Then the following holds: 



A 



i/j(x, C(x))n(dx) 



where [i is the unique invariant measure of X. 

Proof. First notice that tp := ip( ■ , • )) is bounded, by Remark 15.51 Then 

T- l E[Y x - Y$] = T-'E £ U[Xf, aXf)) - Jjd^j dt+ (Jjdfi- \\ . 

We know that T _1 E[Y^ — Y^] — > 0, by the argument in Theorem 14.61 Moreover by the first 
conclusion of Lemma 15.61 



T 



r-'E j \4>(x?,axf)) - J tdv )dt^o, 



and the claim follows. 



Corollary 5.8 In addition to the assumptions of Corollary \5. 7| suppose that F is genuinely 
dissipative. Then v is bounded. 
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Proof. Let (Y x > a ,Z x > a ) be the solution of l[Oj> . We know that Y t x ' a = v a {Xf) and Z*> a = 
C a (X£) with i> a Lipschitz uniformly with respect to a and C, a bounded in H* uniformly with 
respect to a. Let ip a = ip( ■ , £ a ( • )). Under the present assumptions we conclude that also the 
maps ip a as well are bounded in H* uniformly with respect to a. 
Computing d(e~ at Y t xa ) we obtain, 



Y*' a = Ee~ aT Y*> a + E 



Jo 



- at r(xndt, 



and for T — ► oo, 



E 



r(xf)dt. 



Subtracting to both sides a 1 f E ip a (x) /j,(dx) we obtain 



a 



4> a (x)fi(dx) 





/•oo 








/ e" * 




Jo 



t[ij a \{x)- [ ^{X)^{d2 

Je 



dt 



< 4cT7 _1 ||^ a ||o 



where the last inequality comes from the second conclusion of Lemma 15.6 



Thus 



v(x). 



Y 



Y 

1 r 



0,Q 



< 8cr/' 



_1 Ht/j" Ho and the claim follows since by construction Yq ,ol — Y ' a 



6 Ergodic Hamilton-Jacobi-Bellman equations 

We briefly show here that if Y X = v(x) is of class Q 1 then the couple (v, A) is a mild solution of 
the following "ergodic" Hamilton-Jacobi-Bellman equation: 

Cv(x) + ij; (x, Vv(x)G) = A, xeE, (6.1) 

Where linear operator C is formally defined by 

Cf (x) = X -Trace {GG*V 2 f (x)) + (Ax, V/ (x)) e ,e* + {F (x) , V/ (x)) E , E * , 

We notice that we can define the transition semigroup (Pt)t>o corresponding to X by the formula 
(|5.3p for all measurable functions (j) : E — > R having polynomial growth, and we notice that C is 
the formal generator of (Pt)t>o- 

Since we are dealing with an elliptic equation it is natural to consider (v, A) as a mild solution 
of equation (|6.1j) if and only if, for arbitrary T > 0, v(x) coincides with the mild solution u(t, x) 
of the corresponding parabolic equation having v as a terminal condition: 

( du{t, x) + ^ ^ x ) + ^ ( X; Vn (t, x) G) - A = 0, i £ [0, T] , x & E, 

dt (6.2) 
, u(T,x) = v(x), x £ E. 

Thus we are led to the following definition (see also |14j): 

Definition 6.1 A pair (v, A) (v : E — > R and A £ Rj zs a miW solution of the Hamilton-Jacobi- 
Bellman equation \6.1\) if the following are satisfied: 

1. vEG 1 (E,R); 



16 



2. there exists C > such that \Vv (x) h\ < C \h\ E (l + \x\gj for every x,h G E and some 
positive integer k; 

3. forO<t<Tandx£ E, 

v(x) = P T _ t [v] (x) + ^ (P s ^ t [^(-, Vv (•) G)\ (x) - A) ds. (6.3) 

In the right-hand side of f|6.3|) we notice occurrence of the term (•) G, which is not well 
defined as a function E — > H*, since G is not required to map H into E. The situation is similar 
to Remark l5.41 In general, for x £ E, Vv(x)G^ is only defined for £ € Ho- In (16.3H it is implicitly 
required that, P-a.s. and for almost every t, the operator £ — > Vv(X x )G£ can be extended to a 
bounded linear operator defined on the whole H. Noting that 

P t [V(-, (•) G)\ (x) =Ei;(X?,Vv (X?) G) 

the equation f|6.3|) is now meaningful. 

Using the results for the parabolic case, see }17], we get existence of the mild solution of 
equation (|6.1j) whenever we have proved that the function v in Theorem 14.41 is differ entiable. 

Theorem 6.1 Assume that Hypotheses \ 3.1{ and \5.2\ hold. Moreover assume that F is of 
class Q l (E,E) with VF bounded on bounded subsets of E and ip is of class G 1 (E x E,*,E). 

Then (v,X) is a mild solution of the Hamilton-Jacobi-Bellman equation ^6.1\) . 

Conversely, if(v, A) is a mild solution of \6.1\) then, setting Y x = v(X'f) and Z x = Vv(Xf)G, 
the triple (Y X ,Z X ,X) is a solution of the EBSDE [Jlp . 

7 Optimal ergodic control 

Assume that Hypothesis 13.11 holds and let X x denote the solution to equation (|3.ip . Let U be 
a separable metric space. We define a control u as an (^t)-progressively measurable [/-valued 
process. The cost corresponding to a given control is defined in the following way. We assume 
that the functions R : U — > H* and L : E x U — > R are measurable and satisfy, for some constant 
c> 0, 

\R(u)\ < c, \L(x, u)\ < c, \L(x,u) — L(x',u)\ < c\x — x'\, u£U,x,x'£E. (7.1) 
Given an arbitrary control u and T > 0, we introduce the Girsanov density 

p% = ex P n R(u s )dW s -^J^ \R(u s )\Lds^j 

and the probability P^ = p^.P on Tt- The ergodic cost corresponding to u and the starting 
point x G E is 

J(x,u) = limsup^E u ' T I L(X x ,u s )ds, (7.2) 

T^oo T Jq 

where E n,T denotes expectation with respect to P^,. We notice that W" = W t — J * R{u s )ds is a 
Wiener process on [0, T] under P" and that 

dX x = {AX? + F(Xf))dt + G(dW t u + R{u t )dt), t e [0, T] 
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and this justifies our formulation of the control problem. Our purpose is to minimize the cost 
over all controls. 

To this purpose we first define the Hamiltonian in the usual way 

ip(x,z) = inf {L(x,u) + zR(u)\, x G E, z 6 E*, (7.3) 

and we remark that if, for all x, z, the infimum is attained in (|7.3p then there exists a measurable 
function 7 : E x E* — > U such that 

ip(x, z) = 7(x, z)) + zR(j(x, z)). 



This follows from an application of Theorem 4 of [19] . 

We notice that under the present assumptions tp is a Lipschitz function and , 0(-, 0) is bounded 
(here the fact that R depends only on u is used). So if we assume Hypotheses 13.11 and 13.41 then 
in Theorem 14.41 we have constructed, for every x € E, a triple 

(Y*,Z*,\) = (v(X*),aX*),\) (7.4) 

solution to the EBSDE P~T|) . 

Theorem 7.1 Assume that Hypotheses 13.11 Iff.^l anaT3.5l /to/rf. and f/iaf |7. i[) /10/cfe as lye/L 



Moreover suppose that, for some x € E, a triple (Y,Z,X) verifies P-a.s. equation |^.i| ) /or 
all < t < T, where Y is a progressively measurable continuous process, Z is a process in 
L^ loc (n-L 2 (0,oo;E*)) andXeM. Finally assume that there exists c x > (that may depend on 
x) such that P-a.s. 

\Y t \ < c x (\Xf\ + 1), for allt>0. 

Then the following holds: 

(i) For arbitrary control u we have J(x,u) > X = A, and the equality holds if and only if 
L(Xf,u t ) + Z t R(ut) = ip(X£ ,Z t ), P-a.s. for almost every t. 

(ii) If the infimum is attained in J 7. 5] ) then the control u~t = l(Xf, Z t ) verifies J(x,u) = X. 
In particular, for the solution |7.^[) mentioned above, we have: 



(Hi) For arbitrary control u we have J(x,u) = X if and only if L{Xf,ut) + C{Xf)R{ut) = 
ip(Xf X(Xf)), P-a.s. for almost every t. 

(iv) If the infimum is attained in (7.3) then the control ut = y(Xf,£(Xf)) verifies J(x,u) = X. 

Remark 7.2 1. The equality A = A clearly follows from Theorem 14.61 

2. Points (Hi) and (iv) are immediate consequences of (i) and (ii). 

3. The conclusion of point (iv) is that there exists an optimal control in feedback form, with 
the optimal feedback given by the function x >— > j(x,^(x)). 

4. Under the conditions of Theorem 16.11 the pair (v,X) occurring in (|7.4p is a mild solution 
of the Hamilton- Jacobi-Bellman equation (|6.ip . 



5. It follows from the proof below that if limsup is changed into liminf in the definition 
(j7.2p of the cost, then the same conclusions hold, with the obvious modifications, and the 
optimal value is given by A in both cases. 
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Proof. As (Y, Z, A) is a solution of the ergodic BSDE, we have 

-dY t = [iP(Xf,Z t )-X]dt-Z t dWt 

= [iP(X? , Z t ) - X]dt - Z t dW t u - Z t R(u t )dt, 

from which we deduce that 

A = ^ T [Y T -Y ]+^ T ^ £^(Xf,Z t )-Z t r(u t )-L(Xf,Z t )]dt 

^E«> T J L(X?,Z t )dt. 



Thus 



1e«> t J L(Xf, Z t )dt > ^E U > T [Y - Y T ] + A. 

But by (|3.6p we have 

|E"' T Y T | < cK u ' T (\X^\ + 1) < c(l + \x\). 
Consequently T _1 E"' T [Yo -Y T ]^0, and 

lim sup ^E"' T f L(Xf, Z t )dt > A. 

Similarly, if L(X*,u t ) + Z t R(u t ) = ^{Xf, Z t ), 

iE"' T £ L(Xf, Z t )dt = ^ T [Y - Y T ] + A, 

and the claim holds. ■ 

8 Uniqueness 

We wish now to adapt the argument in [15] in order to obtain uniqueness of markovian solutions 
to the EBSDE. This will be done by a control thoretic interpretation the requires that the Markov 
process related to the state equation with continuous feedback enjoys recurrence properties. In 
this section we assume 

E = H and F is bounded. (8.1) 
We recall here a result due to [23] on recurrence of solution to SDEs. 
Theorem 8.1 Consider 

dX t = (AX t + g{X t ))dt + GdW t . (8.2) 

where g : H — > H is bounded and weakly continuous (that if x — > (£,g(x)) is continuous for all 
£ G H). Let 

Q t = f e sA GG*e sA *ds. 
Jo 

and assume the following 

1. sup i>0 Trace (Qt) < oo; 

2. Qt is injective for t > 0; 
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3. e tA {H) C {Q t ) 1/2 {H) for t > 0; 

4. f* \Qs 1/2 e sA \ds < 00 for t > 0; 

5. there exists j3 > such that Jq s~@ Trace (S(s)S(s)*) ds < 00 for t > 0. 

Then, for all T > 0, equation h8. 2\) admits a martingale solution on [0, T], unique in law. The 
associated transition probabilities P(t, x,T, ■) on H (0 < t < T, x € H) identify a recurrent 
Markov process on [0, 00). 

Consider now the ergodic control problem with state equation: 

dX^ u = {AX?> U + F(X?' U ) + GR{u t ))dt + GdW t , X%> u = x, 

and cost 

1 f T 
limsup — E / l(X s ,u s )ds 
T^oo T Jo 

where R : U — ► S is continuous and bounded. 

We restrict ourselves to the class of controls given by continuous feedbacks, i.e. given ar- 
bitrary continuous u : H —> U (called feedback) we define the corresponding trajectory as the 
solution of 

dX*> u = (AX*' U + F(X*> u ))dt + G(R(u(X*> u ))dt + dW t ), X^ x = x. 

We notice that for all T > there exists a weak solution X x,u of this equation, and it is unique 
in law. 

We set as usual 

tp(x, z) = inf {L(x, u) + zR(u)\, 
and assume that ip is continuous and there exists a continuous 7 : H x H — > U such that 

ip(x, z) = L(x, 7(x, z)) + zR(j(x, z)). 

Theorem 8.2 Suppose \8. 1\) and suppose that the assumptions of Theorem \8.1\ hold. Let (v, A) 

with v : H — * M continuous, £ : H — * R continuous, and A a real number satisfy the following 
conditions: 

1. \v[x)\ < c\x\; 

2. for an arbitrary filtered probability space with a Wiener process (Cl, T, {^Ft}t>o, {W*}t>o) 
and for any solution of 

dX t = (AX t + F{X t ))dt + GdW t , t € [0, T], 

setting Y t = v(X t ), Z t = C,(X t ), we have 

-dY t = mx t , Z t ) - X]dt - Z t dW t t E [0, T]. 
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Let 

r r T = inf{s € [0,7] : \X^ X \ < r}, 
with the convention tJ = T if the indicated set is empty, and 

T 

J(x,u) =limsuplimsupE / ' [ip(Xp u , u(X*> u )) - X]ds. 

r->0 T^oo JO 

Then 

v(x) = inf J(x, u), 

u 

where the infimum (that is a minimum) is taken over all continuous feedbacks u. 

Proof. Let u : H — > U be continuous. We notice that X x ' u solves on [0, 7]: 

dX*' u = (AX*' U + F{X*' u ))ds + GdW t u , t e [0,7], 

where W t = Jj R(u(X?> u )dr + W t is a Wiener process on [0, T] under a suitable probability P u > . 
Therefore Y t = v(X^ u ), Z t = Q{X*' U ) satisfy: 

-dY t = [tP(X?' u ,u(X?> u )) - X]dt - Z t R(u(X?' u ))]dt - Z t dW t . 

Integrating in [0,^] we get 

T 

v(x) = E(v(X% u ))+E f r mx^,u(X^)) - A - Z s R(X^)]ds. 

Thus, 



o 



Now 



T 

v(x) < E(v(X*?))+E r [L(X^,u(X^)) - \}ds. (8.3) 

Jo 

\E(v(X% u ))\ < cE\X% u \ < cr + (E(\X^ u \ 2 ))^ 2 (F(tJ = r)) 1 / 2 

< cr + c(P(r r T = r)) 1/2 

Notice that P(r r T = r) = P(mf t6 [ 0jT ] \Xt\ > r )> where X is the Markov process on the whole 
[0, +00) corresponding to the equation (|8.2|) with g = F(-) + GR(u(-)). 

Since X is recurrent, for all r > it holds P^nf^myi \X t \ > r) — > as T — > 00. Thus 

lim sup lim sup |E(v (X x ^))\ — > 0. 

Hence, 

T 
/'TT 

v(x) < lim sup lim sup E / *" [l(X%> u , u(X*' u )) - X]ds. 

r-*0 T^oo Jo 

The proof is completed noticing that if u is chosen as u(x) = 7(2;, ((x)) then the above inequality 
becomes an equality. ■ 

This result combines with Theorems 14.61 and 16.11 to give the following 

Corollary 8.3 Suppose that all the assumptions of Theorems \4-6\tfTl\ and [KM hold. Then (v,X) 
is the unique mild solution of the Hamilton- J acobi- Bellman equation Ii6.1\) satisfying \v(x)\ < 
c\x\. 
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9 Application to ergodic control of a semilinear heat equation 



In this section we show how our results can be applied to perform the synthesis of the ergodic 
optimal control when the state equation is a semilinear heat equation with additive noise. More 
precisely, we treat a stochastic heat equation in space dimension one, with a dissipative nonlinear 
term and with control and noise acting on a subinterval. We consider homogeneous Dirichlet 
boundary conditions. 

In (fl, F, IP) with a filtration (F t ) t>0 satisfying the usual conditions, we consider, for t G [0,T] 
and £ G [0, 1] , the following equation 

' d t x u (t, £) = [^x u (t, o + f (£, x u (t, 0) + x[a, b] (0« (*. 0] dt + X[ a,b] (Z)w (t, £) dt, 

X u (t,0)=X u (t,l)=0, 

{ x u (t,o = x (0, 

(9-1) 

where X[a,b] is the indicator function of [a, b] with < a < b < 1; W (t, £) is a space-time white 

noise on [0, T] x [0,1]. 

We introduce the cost functional 

J(x,u) = Umsup^E [ T A ds, (9.2) 

T^oo J- JO JO 

where fi is a finite Borel measure on [0, 1] . An admissible control u (t, £) is a predictable process 
such that for all t > 0, and P-a.s. u (t, ■) £ U := {v e C ([0, 1]) : \v (£)| < <5}. We denote by W the 
set of such admissible controls. We wish to minimize the cost over U, adopting the formulation 
of Section [71 i.e. by a change of probability in the form of (|7.2p . The cost introduced in (|9.2p is 
well defined on the space of continuous functions on the interval [0, 1] , but for an arbitrary fi it 
is not well defined on the Hilbert space of square integrable functions. 
We suppose the following: 

Hypothesis 9.1 1. f : [0, 1] x R — > R is continuous and for every £ G [0,1], /(£, •) is 
decreasing. Moreover there exist C > and m > such that for every £ G [0, 1] , x G R, 

1/ (£, x) | < C(l + M) m , / (0, x) = / (1, x) = 0. 

/ : [0, 1] x R x [—5,5] — > R is continuous and bounded, and Z(£, -,u) is Lipschitz continuous 
uniformly with respect to £ G [0,1], it € [—5,5]. 

3. xo GC ([0,1]), x (0) =x (l) = 0. 

To rewrite the problem in an abstract way we set -ff = 5 = L 2 (0, 1) and i£ = Co ([0, 1]) = {y G 
C ([0, 1]) : j/(0) = y(l) = 0}. We define an operator A in £ by 



D (A) = {yeC 2 ([0, 1]) : ,/. u" € ( o I [0. 1 ] ) } • {Ay) (£) = J^y (£) for y G D (A) 



We notice that A is the generator of a Co semigroup in E, admitting and extension to H, and 
l e L(£ E) — e ~ l see ' ^ or instance, Theorem 11.3.1 in [7J. As a consequence, A + F + I is 
dissipative in 

We set, for x G E, £ G [0, 1], z G 3, u G U, 

F (x) (£) = / (£, x (£)) , (Gz) (£) = X [aM (0 z(0. ^ «) = / * (£, x (0 , « (0) A* (<£) , (9.3) 

Jo 



22 



and let R denote the canonical imbedding of C([0, 1]) in L 2 (0, 1). 

Finally {Wt,t > 0} is a cylindrical Wiener process in H with respect to the filtration (^t) t >o 

It is easy to verify that Hypotheses 13.11 and 13 .41 are satisfied (for the proof of point 4 in Hypothesis 
13.11 and of Hypothesis E3 see again [7] Theorem 11.3.1.). 
Moreover, see for instance [5], for some C > 0, 



\e tA \ 



L{H>E) <Cr"\ *e(o,l], 



thus Hypothesis 13.51 holds. 

Also Hypothesis is satisfied by taking ~ = {/ G C ([0, 1]) : f(a) = f(b) = 0}. 

Clearly the controlled heat equation (|9.ip can now be written in abstract way in the Banach 
space E as 



dx x ,u = ^ Ax x ,u + p ( X *°' u )] dt + GRu t dt + GdW* t G [i, T 

A — X , 



and the results of the previous sections can be applied to the ergodic cost (|9.2p (reformulated 
by a change of probability in the form of (|7.2|) ). 

In particular if we define, for all x G C ([0, 1]), z G L 2 (0, 1), u G U (identifying L 2 (0, 1) with its 
dual) 

ii>{x,z) = wilj mx(0M0Md0+ J ztiwoda 

then there exist v : E — > 1R Lipschitz continuous and with ?J(0) = 0, C : E — > H* measurable and 
A G R such that if X x ° = X x °'° is the solution of equation |E3|) then (v(X x °),((X x °),~\) is a 
solution of the EBSDE (|4.ip and the characterization of the optimal ergodic control stated in 
Theorem 17.11 holds (and A is unique in the sense of Theorem I4.6j) . 



Moreover if / is of class C 1 (]R) (consequently F will be of class Q 1 ^, E)) and if) is of class 
Q 1 {E x H*,.E) then by Theorem 15.11 v is of class Q 1 (E,E) and, by Theorem 16.11 it is a mild 
solution of the ergodic HJB equation (|6.ip and it holds ( = VvG. 

Let us then consider the particular case in which [a, b] = [0,1], f(x,^) = f(x) is of class C 1 
with derivative having polynomial growth, and satisfies /(0) = 0, [f(x + h) — f(x)]h < — c\h\ 2+e 
for suitable c, e > and all x,h G M (for instance, f(x) = — x 3 ). In that case the Kolmogorov 
semigroup corresponding to the process X x ° is strongly Feller, see [5] and [18] . and it is easy to 
verify that F is genuinely dissipative (see Definition l5.2p . Moreover we can choose Ho = Co([0, 1]) 
and it turns out that ip is bounded on each set E x B, where B is any ball of HJ. Thus the 
claims of Corollaries 15.71 and 15.81 hold true, and in particular v is bounded. 

Finally if we assume that fi is Lebesgue measure and / is bounded and Lipschitz we can choose 
E = 'E = 'Eo = H = L 2 (0, 1). Then the assumptions of Theorem 18.11 are satisfied and we can 
apply Theorem 18.21 to characterize the function v. In particular if / is of class C 1 (1R) and ip is 
of class Q l {H x E*,H) then v is the unique mild solution of the ergodic HJB equation (|6.ip . 
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